[WIP] Demonstration provider #4988
Conversation
|
|
||
|
|
||
| @timed | ||
| def load_demonstration( |
There was a problem hiding this comment.
was demo_loader.load_demonstration
| return trajectories | ||
|
|
||
| @staticmethod | ||
| def _get_demo_files(path: str) -> List[str]: |
There was a problem hiding this comment.
from demo_loader.get_demo_files
| def get_behavior_spec(self) -> BehaviorSpec: | ||
| return self._behavior_spec | ||
|
|
||
| def pop_trajectories(self) -> List[DemonstrationTrajectory]: |
There was a problem hiding this comment.
Need to add docstrings here. But the idea is that GAIL, etc could be converted to use pop_trajectories() directly. Then if we want DemonstrationProviders to be able to load new demonstrations on the fly, the logic can be kept in the DemonstrationProvider and the consumer doesn't need to know about it, it just gets a fresh batch of trajectories.
| from mlagents.trainers.trajectory import ObsUtil | ||
|
|
||
|
|
||
| class DemonstrationExperience(NamedTuple): |
There was a problem hiding this comment.
These are trimmed down versions of AgentExperience and Trajectory classes, based on what's currently in demo_loader.
There was a problem hiding this comment.
Hmm, I feel like we shouldn't duplicate the conversion code here between AgentExperience and Trajectory (esp. with the teammate observations coming in, it becomes quite a fat function - and at some point I imagine we'll have teammate demonstrations as well).
Wonder if we can have a BaseAgentExperience be the base class that is used here and in the AgentProcessor, and have the AgentExperience (PolicyAgentExperience?) inherit from it? Or some other way of composing these two.
|
Shelving this for now, since research won't be able to make use of it for a while, and we're still not sure on the trajectory/experience handling. |
2 participants
Proposed change(s)
Very rough WIP to convert GAIL and BC to use a new DemonstrationProvider interface.
The eventual goal (beyond the scope of this PR) is to let users define their own DemonstrationProvider interface in a plugin and use that instead of the given LocalDemonstrationProvider.
In this PR:
TODO
Useful links (Github issues, JIRA tickets, ML-Agents forum threads etc.)
https://jira.unity3d.com/browse/MLA-1734
Types of change(s)
Checklist
Other comments